{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Debugging a pipeline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "River encourages users to make use of pipelines. The biggest pain point of pipelines is that it can be hard to understand what's happening to the data, especially when the pipeline is complex. Fortunately the `Pipeline` class has a `debug_one` method that can help out.\n",
    "\n",
    "Let's look at a fairly complex pipeline for predicting the number of bikes in 5 bike stations from the city of Toulouse. It doesn't matter if you understand the pipeline or not; the point of this notebook is to learn how to introspect a pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-12-04T17:50:33.792436Z",
     "iopub.status.busy": "2023-12-04T17:50:33.792067Z",
     "iopub.status.idle": "2023-12-04T17:50:43.261751Z",
     "shell.execute_reply": "2023-12-04T17:50:43.261453Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "30000 MAE: 13.328051\n",
      "60000 MAE: 7.824087\n",
      "90000 MAE: 6.003909\n",
      "120000 MAE: 5.052855\n",
      "150000 MAE: 4.496826\n",
      "180000 MAE: 4.140702\n"
     ]
    }
   ],
   "source": [
    "import datetime as dt\n",
    "from river import compose\n",
    "from river import datasets\n",
    "from river import feature_extraction\n",
    "from river import linear_model\n",
    "from river import metrics\n",
    "from river import preprocessing\n",
    "from river import stats\n",
    "from river import stream\n",
    "\n",
    "\n",
    "X_y = datasets.Bikes()\n",
    "X_y = stream.simulate_qa(X_y, moment='moment', delay=dt.timedelta(minutes=30))\n",
    "\n",
    "def add_time_features(x):\n",
    "    return {\n",
    "        **x,\n",
    "        'hour': x['moment'].hour,\n",
    "        'day': x['moment'].weekday()\n",
    "    }\n",
    "\n",
    "model = add_time_features\n",
    "model |= (\n",
    "    compose.Select('clouds', 'humidity', 'pressure', 'temperature', 'wind') +\n",
    "    feature_extraction.TargetAgg(by=['station', 'hour'], how=stats.Mean()) +\n",
    "    feature_extraction.TargetAgg(by='station', how=stats.EWMean())\n",
    ")\n",
    "model |= preprocessing.StandardScaler()\n",
    "model |= linear_model.LinearRegression()\n",
    "\n",
    "metric = metrics.MAE()\n",
    "\n",
    "questions = {}\n",
    "\n",
    "for i, x, y in X_y:\n",
    "    # Question\n",
    "    is_question = y is None\n",
    "    if is_question:\n",
    "        y_pred = model.predict_one(x)\n",
    "        questions[i] = y_pred\n",
    "    \n",
    "    # Answer\n",
    "    else:\n",
    "        metric.update(y, questions[i])\n",
    "        model.learn_one(x, y)\n",
    "    \n",
    "        if i >= 30000 and i % 30000 == 0:\n",
    "            print(i, metric)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by looking at the pipeline. You can click each cell to display the current state for each step of the pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-12-04T17:50:43.263530Z",
     "iopub.status.busy": "2023-12-04T17:50:43.263338Z",
     "iopub.status.idle": "2023-12-04T17:50:43.278572Z",
     "shell.execute_reply": "2023-12-04T17:50:43.278309Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div class=\"river-component river-pipeline\"><details class=\"river-component river-estimator\"><summary class=\"river-summary\"><pre class=\"river-estimator-name\">add_time_features</pre></summary><code class=\"river-estimator-params\">\n",
       "def add_time_features(x):\n",
       "    return {\n",
       "        **x,\n",
       "        'hour': x['moment'].hour,\n",
       "        'day': x['moment'].weekday()\n",
       "    }\n",
       "\n",
       "</code></details><div class=\"river-component river-union\"><details class=\"river-component river-estimator\"><summary class=\"river-summary\"><pre class=\"river-estimator-name\">['clouds', [...]</pre></summary><code class=\"river-estimator-params\">Select (\n",
       "  clouds\n",
       "  humidity\n",
       "  pressure\n",
       "  temperature\n",
       "  wind\n",
       ")\n",
       "</code></details><details class=\"river-component river-estimator\"><summary class=\"river-summary\"><pre class=\"river-estimator-name\">y_mean_by_station_and_hour</pre></summary><code class=\"river-estimator-params\">TargetAgg (\n",
       "  by=['station', 'hour']\n",
       "  how=Mean ()\n",
       "  target_name=\"y\"\n",
       ")\n",
       "</code></details><details class=\"river-component river-estimator\"><summary class=\"river-summary\"><pre class=\"river-estimator-name\">y_ewm_0.5_by_station</pre></summary><code class=\"river-estimator-params\">TargetAgg (\n",
       "  by=['station']\n",
       "  how=EWMean (\n",
       "    fading_factor=0.5\n",
       "  )\n",
       "  target_name=\"y\"\n",
       ")\n",
       "</code></details></div><details class=\"river-component river-estimator\"><summary class=\"river-summary\"><pre class=\"river-estimator-name\">StandardScaler</pre></summary><code class=\"river-estimator-params\">StandardScaler (\n",
       "  with_std=True\n",
       ")\n",
       "</code></details><details class=\"river-component river-estimator\"><summary class=\"river-summary\"><pre class=\"river-estimator-name\">LinearRegression</pre></summary><code class=\"river-estimator-params\">LinearRegression (\n",
       "  optimizer=SGD (\n",
       "    lr=Constant (\n",
       "      learning_rate=0.01\n",
       "    )\n",
       "  )\n",
       "  loss=Squared ()\n",
       "  l2=0.\n",
       "  l1=0.\n",
       "  intercept_init=0.\n",
       "  intercept_lr=Constant (\n",
       "    learning_rate=0.01\n",
       "  )\n",
       "  clip_gradient=1e+12\n",
       "  initializer=Zeros ()\n",
       ")\n",
       "</code></details></div><style scoped>\n",
       ".river-estimator {\n",
       "    padding: 1em;\n",
       "    border-style: solid;\n",
       "    background: white;\n",
       "    max-width: max-content;\n",
       "}\n",
       "\n",
       ".river-pipeline {\n",
       "    display: flex;\n",
       "    flex-direction: column;\n",
       "    align-items: center;\n",
       "    background: linear-gradient(#000, #000) no-repeat center / 1.5px 100%;\n",
       "}\n",
       "\n",
       ".river-union {\n",
       "    display: flex;\n",
       "    flex-direction: row;\n",
       "    align-items: center;\n",
       "    justify-content: center;\n",
       "    padding: 1em;\n",
       "    border-style: solid;\n",
       "    background: white;\n",
       "}\n",
       "\n",
       ".river-wrapper {\n",
       "    display: flex;\n",
       "    flex-direction: column;\n",
       "    align-items: center;\n",
       "    justify-content: center;\n",
       "    padding: 1em;\n",
       "    border-style: solid;\n",
       "    background: white;\n",
       "}\n",
       "\n",
       ".river-wrapper > .river-estimator {\n",
       "    margin-top: 1em;\n",
       "}\n",
       "\n",
       "/* Vertical spacing between steps */\n",
       "\n",
       ".river-component + .river-component {\n",
       "    margin-top: 2em;\n",
       "}\n",
       "\n",
       ".river-union > .river-estimator {\n",
       "    margin-top: 0;\n",
       "}\n",
       "\n",
       ".river-union > .river-component {\n",
       "    margin-top: 0;\n",
       "}\n",
       "\n",
       ".river-union > .pipeline {\n",
       "    margin-top: 0;\n",
       "}\n",
       "\n",
       "/* Spacing within a union of estimators */\n",
       "\n",
       ".river-union > .river-component + .river-component {\n",
       "    margin-left: 1em;\n",
       "}\n",
       "\n",
       "/* Typography */\n",
       "\n",
       ".river-estimator-params {\n",
       "    display: block;\n",
       "    white-space: pre-wrap;\n",
       "    font-size: 110%;\n",
       "    margin-top: 1em;\n",
       "}\n",
       "\n",
       ".river-estimator > .river-estimator-params,\n",
       ".river-wrapper > .river-details > river-estimator-params {\n",
       "    background-color: white !important;\n",
       "}\n",
       "\n",
       ".river-wrapper > .river-details {\n",
       "    margin-bottom: 1em;\n",
       "}\n",
       "\n",
       ".river-estimator-name {\n",
       "    display: inline;\n",
       "    margin: 0;\n",
       "    font-size: 110%;\n",
       "}\n",
       "\n",
       "/* Toggle */\n",
       "\n",
       ".river-summary {\n",
       "    display: flex;\n",
       "    align-items:center;\n",
       "    cursor: pointer;\n",
       "}\n",
       "\n",
       ".river-summary > div {\n",
       "    width: 100%;\n",
       "}\n",
       "</style></div>"
      ],
      "text/plain": [
       "Pipeline (\n",
       "  FuncTransformer (\n",
       "    func=\"add_time_features\"\n",
       "  ),\n",
       "  TransformerUnion (\n",
       "    Select (\n",
       "      clouds\n",
       "      humidity\n",
       "      pressure\n",
       "      temperature\n",
       "      wind\n",
       "    ),\n",
       "    TargetAgg (\n",
       "      by=['station', 'hour']\n",
       "      how=Mean ()\n",
       "      target_name=\"y\"\n",
       "    ),\n",
       "    TargetAgg (\n",
       "      by=['station']\n",
       "      how=EWMean (\n",
       "        fading_factor=0.5\n",
       "      )\n",
       "      target_name=\"y\"\n",
       "    )\n",
       "  ),\n",
       "  StandardScaler (\n",
       "    with_std=True\n",
       "  ),\n",
       "  LinearRegression (\n",
       "    optimizer=SGD (\n",
       "      lr=Constant (\n",
       "        learning_rate=0.01\n",
       "      )\n",
       "    )\n",
       "    loss=Squared ()\n",
       "    l2=0.\n",
       "    l1=0.\n",
       "    intercept_init=0.\n",
       "    intercept_lr=Constant (\n",
       "      learning_rate=0.01\n",
       "    )\n",
       "    clip_gradient=1e+12\n",
       "    initializer=Zeros ()\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As mentioned above the `Pipeline` class has a `debug_one` method. You can use this at any point you want to visualize what happen to an input `x`. For example, let's see what happens to the last seen `x`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-12-04T17:50:43.280147Z",
     "iopub.status.busy": "2023-12-04T17:50:43.280048Z",
     "iopub.status.idle": "2023-12-04T17:50:43.290636Z",
     "shell.execute_reply": "2023-12-04T17:50:43.290346Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0. Input\n",
      "--------\n",
      "clouds: 88 (int)\n",
      "description: overcast clouds (str)\n",
      "humidity: 84 (int)\n",
      "moment: 2016-10-05 09:57:18 (datetime)\n",
      "pressure: 1,017.34000 (float)\n",
      "station: pomme (str)\n",
      "temperature: 17.45000 (float)\n",
      "wind: 1.95000 (float)\n",
      "\n",
      "1. add_time_features\n",
      "--------------------\n",
      "clouds: 88 (int)\n",
      "day: 2 (int)\n",
      "description: overcast clouds (str)\n",
      "hour: 9 (int)\n",
      "humidity: 84 (int)\n",
      "moment: 2016-10-05 09:57:18 (datetime)\n",
      "pressure: 1,017.34000 (float)\n",
      "station: pomme (str)\n",
      "temperature: 17.45000 (float)\n",
      "wind: 1.95000 (float)\n",
      "\n",
      "2. Transformer union\n",
      "--------------------\n",
      "    2.0 Select\n",
      "    ----------\n",
      "    clouds: 88 (int)\n",
      "    humidity: 84 (int)\n",
      "    pressure: 1,017.34000 (float)\n",
      "    temperature: 17.45000 (float)\n",
      "    wind: 1.95000 (float)\n",
      "\n",
      "    2.1 TargetAgg\n",
      "    -------------\n",
      "    y_mean_by_station_and_hour: 7.89396 (float)\n",
      "\n",
      "    2.2 TargetAgg1\n",
      "    --------------\n",
      "    y_ewm_0.5_by_station: 11.80372 (float)\n",
      "\n",
      "clouds: 88 (int)\n",
      "humidity: 84 (int)\n",
      "pressure: 1,017.34000 (float)\n",
      "temperature: 17.45000 (float)\n",
      "wind: 1.95000 (float)\n",
      "y_ewm_0.5_by_station: 11.80372 (float)\n",
      "y_mean_by_station_and_hour: 7.89396 (float)\n",
      "\n",
      "3. StandardScaler\n",
      "-----------------\n",
      "clouds: 1.54778 (float)\n",
      "humidity: 1.16366 (float)\n",
      "pressure: 0.04916 (float)\n",
      "temperature: -0.51938 (float)\n",
      "wind: -0.69426 (float)\n",
      "y_ewm_0.5_by_station: 0.19640 (float)\n",
      "y_mean_by_station_and_hour: -0.27110 (float)\n",
      "\n",
      "4. LinearRegression\n",
      "-------------------\n",
      "Name                         Value      Weight     Contribution  \n",
      "                 Intercept    1.00000    9.19960        9.19960  \n",
      "      y_ewm_0.5_by_station    0.19640    9.19349        1.80562  \n",
      "                  humidity    1.16366    1.01680        1.18320  \n",
      "               temperature   -0.51938   -0.41575        0.21593  \n",
      "                      wind   -0.69426   -0.03810        0.02645  \n",
      "                  pressure    0.04916    0.18321        0.00901  \n",
      "y_mean_by_station_and_hour   -0.27110    0.19553       -0.05301  \n",
      "                    clouds    1.54778   -0.32838       -0.50827  \n",
      "\n",
      "Prediction: 11.87854\n"
     ]
    }
   ],
   "source": [
    "print(model.debug_one(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The pipeline does quite a few things, but using `debug_one` shows what happens step by step. This is really useful for checking that the pipeline is behaving as you're expecting it too. Remember that you can `debug_one` whenever you wish, be it before, during, or after training a model."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
